Theoretical and practical limitations usually constrain the achievable resolution of any imaging device. While higher quality images may result from more expensive imaging systems, often it is desirable to increase the resolution of images previously captured under non-ideal situations. For instance, enhancing the quality of a video sequence captured by surveillance cameras in a crime scene is an example of these situations. The basic idea behind super-resolution is the fusion of a sequence of low-resolution noisy blurred images to produce a higher-resolution image. Early works on super-resolution showed that it is the aliasing effects in the low-resolution images that enable the recovery of the high-resolution fused image, provided that a relative subpixel motion exists between the undersampled input images. In contrast to the clean but practically naive frequency-domain description of super-resolution proposed in the early works on this subject, in general, super-resolution is a computationally complex and numerically ill-posed problem in many instances. In recent years, more sophisticated super-resolution methods have been developed.
Previous approaches have used super-resolution applied on an image sequence, producing a sequence of super-resolution images. At time point t, a super-resolution result is desired that fuses the causal images at times t, t−1, . . . , 1. The approaches were to apply the regular super-resolution on this set of images with the tth frame as a reference, produce the super-resolution output, and repeat this process all over again per each temporal point. This is referred to as the static super-resolution method, since it does not exploit the temporal evolution of the process. The memory and computational requirements for the static process are so taxing as to preclude its direct application to the dynamic case, without highly efficient algorithms.
Previous work on applications to the dynamic case have relied on the information pair to approximate the Kalman filter, but have proved to be computationally expensive.
Previous work considered a causal mode of operation, where the output image at time t0 fuses the information from times t≦t0. This is the appropriate mode of operation when online processing is considered however is not capable of providing optimum image enhancement in non-causal modes of operation.
What is needed is a computationally less complex than the dynamic super-resolution methods, and a means to decompose this problem into two disjoint pieces, without sacrificing quality.
Further, what is needed is a method of simultaneously addressing two common resolution-enhancement problems in digital video/photography that are typically addressed separately, namely, super-resolution and demosaicing. A method is needed for a dynamic super-resolution algorithm for both monochromatic and color input and output sequences.
Improvements are needed in both visual quality (resolution enhancement and color artifact reduction) and computational/memory efficiency. And finally, a non-causal processing mode is needed, where every high-resolution reconstructed image is derived as an optimal estimate incorporating information from all the frames in the sequence.